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Specification 

(54) Title of the Invention : MULTI-MEDIA TERMINAL DEVICE 

AND COMMUNICATIONS 
CONNECTION CONTROL FORMAT 

(57 ) Summary 

Objective: The objective of the present invention is to 
improve the operative efficiency of a user's communications 
connection access with regard to a joint operative system 
constituted by multiple work stations. 

Consti tution : Multi-media communications connection icons 
^?:9) thr „°- u -? 11 < 330 )' the functions of which are designated to 
vary depending on the current connection statuses of 
communis in correspondence to the respective 

terminal devices (participants) engaging in a joint operation, 
are configured, and upon the selection of any of these icons, 
the control program (680 ) ensures multi-media communications 
connection modalities optimally suited for the current status. 

Effects : Connection operations for multi-media 
communications paths can be significantly simplified. 
Claims 

1. A multi-media communications connection control format 
with the following characteristics: In a communications system 
which consists of multiple terminal devices endowed each with 
a window function, multiple communications control icons which 
correspond to the modes of joint operations which are to be 
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executed via said terminal device are d isp layed on a singular 
window generated on the screen of each terminal device, 
whereas the functions of these communications control icons 
are dynamically redesignated in accordance with the functions 
of selected communications control icons, whereas [said 
respective terminals are mutually] connected via 
communications media which accord with the functions of said 
selected communications control icons. 

2. The multi-media communications connection control 
format specified in Claim 1 wherein the functions of said 
communications control icons are redesignated in accordance 
with terminal devices which serve as connected nodes. 

3. The multi-media communications connection control 
format specified in Claim 1 or 2 wherein, in a case where 
terminal devices which serve as connected nodes have been 
designated, the addresses of the terminal devices of said 
nodes are memorized, and wherein, in a case where any of the 
aforementioned communications control icons is subsequently 
selected in relation to said node terminal devices, the 
respective node terminal devices are connected via 
communications media specified by the functions of said icons. 

4. The multi-media communications connection control 
format specified in Claim 1, 2, or 3 wherein node display 
icons which correspond to terminal devices which serve as 
connection nodes are displayed on the screen of each of said 
terminal devices and wherein node terminal devices and 
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communications media which are to be subjected to connection 
control routines are designated by selecting respectively 
proper ones from among the aforementioned node display icons 
and communications control icons. 

5. The multi-media communications connection control 
format specified in Claim 1, 2, or 3 wherein a window for 
displaying said communications control icons in correspondence 
to the terminal devices which serve as connection nodes is 
displayed on the screen of each of said respective terminal 
devices and wherein, in a case where any of said 
communications control icons has been selected, node terminal 
devices corresponding to the window in which the selected icon 
is being positioned are mutually connected via communications 
media specified by the function of said selected 
communications control icon. 

6. A multi-media communications connection control format 
with the following characteristics: In a communications system 
which consists of multiple terminal devices endowed each with 
a window function, multiple types of communications control 
icons which have been prepared for the connection control of 
one or mutually different multiple types of communications 
media are displayed on-screen, whereas the communications 
media connection control modes to be executed in 
correspondence to said respective communications connection 
control icons are dynamically redesignated, whereas, in a case 
where any of said communications connection control icon has 
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been selected, [a given terminal device is] connected to [an] 
other terminal device (s) via communications media specified by 
the function of said icon. 

7. A multi-media terminal device with the following 
characteristics: In a multi-media terminal device which is 
connected with at least one other terminal device via a 
network, a mechanism which displays multiple types of 
communications connection control icons which have been 
prepared for connection control purposes via one or mutually 
different multiple types of communications media on-screen, an 
icon function control mechanism which dynamically redesignates 
the communications media connection control routines to be 
executed in correspondence to said respective communications 
connection control icons, and an icon selection mechanism are 
configured while it is being connected to said another 
terminal device (s) via the communications media which have 
been specified by the functions of the communications 
connection control icons which have been selected by said 
selection mechanism. 

8. The multi-media terminal device specified in Claim 7 
wherein said icon function control mechanism redesignates the 
functions of said communications control icons in 
correspondence to the terminal devices which serve as 
connection nodes. 

9. The multi-media terminal device specified in Claim 7 
or 8 which additionally possesses a mechanism which, in a case 
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where a terminal device (s) which serves as a connection 
node(s) has been selected, memorizes the address (es) of said 
terminal device (s) and wherein, in a case where any of said 
communications control icons is subsequently selected in 
relation to said node terminal device (s), a connection with 
said node terminal device is established via the 
communications media which have been specified by the function 
of said icon with reference to the address (es) memorized in 
said mechanism. 

10. The multi-media terminal device specified in Claim 7, 
8 or 9 wherein said icon display mechanism displays a node 
display icon which corresponds to the terminal device which 
serves as a connection node and wherein a node terminal 
device (s) which is to be subjected to the connection control 
routine and communications media are designated by selecting 
said node display icon and said communications control icon. 

11. The multi-media terminal device specified in Claim 7, 
8 or 9 wherein said icon display mechanism generates a window 
for displaying said communications control icon corresponding 
to the terminal device (s) which serves as a connection node(s) 
and wherein, in a case where a communications control icon has 
been selected, connection is established with the node 
terminal device which corresponds to the window in which said 
icon is being displayed via the communications media which 
have been specified by the function of said selected 
communications control icon. 
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Detailed explanation of the invention 
[0001] 

(Industrial application fields of the invention) 

The present invention concerns a multi-media terminal 
device as well as a communications connection format. More 
specifically, it concerns a terminal device to be used for a 
multi-media communications system wherein multiple terminal 
devices (or work stations) are mutually connected via a multi- 
media communications path and wherein multiple users which are 
physically apart can engage^ in conferences, planning meetings, 
operations for jointly compiling data, etc. by using commonly 
sha 5^ d data ' animate images of the other party, or voices 
wh i ch a ? e dis P la y ed on their respective terminal screens as 
well as a corresponding communications media connection 
control format. 
[0002] 

(Prior art of the invention) 

As far as multiple-site conference systems which are 
convened by multiple users who communicate with one another 
via conference screens which have been formed on terminals or 
work stations which are mutually connected via communications 
lines are concerned, many formats for connecting the 
respective terminals and modalities for transmitting and 
receiving data have been proposed. Japanese Patent 
Application Publication No. Kokai Hei 3 [ 1991 ] -157044 , for !_ 
example, proposes a format wherein the number of television 



conference participants (i.e., number of terminals) can be 
arbitrarily enlarged. Japanese Patent Application Publication 
No. Kokai Sho 63 [ 1988 ] -1140, furthermore, proposes a format 
for simultaneously controlling voice lines and data lines for 
the purpose of improving the call transmission and cut-off 
operative efficiencies of composite terminals which 
communicate with one another via telephone lines and data 
lines . 
[0003] 

(Problems to be solved by the invention) 

The aforementioned proposals are intended to improve the 
operative efficiencies of line connection control protocols in 
cases where multiple participants engage in electronic joint 
operations via terminals. The objects of connection control, 
however, are limited to telephone machines and computer 
terminals, and therefore, they fail to provide a flexible 
communications media connection control format which can be 
applied to general-purpose multi-media communications purposes 
which involve other communications media. 
[0004] 

One objective of the present invention is to provide a 
terminal device which can be effectively applied to an 
electronic conversation system or joint operation system 
wherein multiple terminal devices are mutually connected via a 
multi-media communications path as well as a corresponding 
communications connection control format. 
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[0005] 

Another objective of the present invention is to provide 
a terminal device, electronic conversation system, or a joint 
operation system the device functions of which can be 
dynamically redesignated in accordance with the communications 
status variations of the communications media. 
[0006] 

Still another objective of the present invention is to 
provide a communications connection control format which 
facilitates communications media switch operations by users. 
[0007] 

(Problems to be solved by the invention) 

The following constitution is provided by the multi-media 
terminal device and communications connection control format 
of the present invention in order to achieve the 
aforementioned objectives: In a communications system which 
consists of multiple terminal devices endowed each with a 
window function, multiple communications control icons which 
correspond to the modes of joint operations which are to be 
executed via said terminal device are displayed on a singular 
window generated on the screen of each terminal device, 
whereas the functions of these communications control icons 
are dynamically redesignated in accordance with the functions 
of selected communications control icons, whereas [said 
respective terminals are mutually] connected via 
communications media which accord with the functions of said 



selected communications control icons. The modalities of the 
"joint operation" as it is being hereby referred to signify 
operation modalities which utilize different communications 
media (e.g., t^ephone conversation, TV telephone conversation 
with reference to video images, electronic conversations which 
utilize telephone and data processing outputs, conversations 
which simultaneously utilize TV telephones and data processing 
outputs, etc. } . 
[0008] 

As far as another aspect of the present invention is 
concerned, multiple types of communications control icons 
which have been prepared for the connection control of one or 
mutually different multiple types of communications media are 
displayed on the respective terminal devices while the 
functions of these icons are being dynamically redesignated 
for the purpose of controlling the connections among multiple 
terminal devices (or work stations) . The aforementioned icon 
functions may, for example, be redesignated by using an icon 
functional status control mechanism in which communications 
media connection control protocols to be executed in response 
to the selections of the respective communications connection 
control icons have been defined. In a desirable application 
example of the present invention, the aforementioned icon 
functions can be redesignated in accordance with connection 
node devices. 
[0009] 
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As far as still another aspect of the present invention 
is concerned, in a case where a connection node- has become 
designated, the address information on the node device is 
memorized, and in a case where a command for altering the 
communications media connection modalities vis-a-vis said node 
device is subsequently issued, the communications path 
communications control routine is executed by using the 
previously memorized address information instead of having to 
redesignate the address of the connection node on the part "of 
the user. 
[0010] 
( Functions) 

As far as the present invention is concerned, a given 
icon is endowed with different functions depending on the 
communications media connection statuses vis-a-vis node 
devices at the time of operation.. In a case where a control 
icon which uses multiple types of communications media A and B 
has been selected, for example, the communications media 
connection control protocols which are executed in response to 
icon operations come to differ mutually between a case where 
said control icon has been selected in a state where [the 
device which has selected said icon has] already been 
connected to the node device via the medium A and a case where 
said control icon has been selected in an utterly unconnected 
state. In the former case, the aforementioned control icon 
serves the function of a connection command icon for one 



medium B, whereas in the latter case, the aforementioned 
control icon serves as a connection command icon for both 
media A and B. 
[0011] 

As far as the present invention is concerned, in a case 
where it has become necessary for users to redesignate the 
types of communications media to be connected to the node 
devices or their connection modalities in the context of 
altering the types or scales of conferences or joint 
operations, a control icon which expresses the preferred 
operation mode to be executed in relation to the node device 
is designated, based on which a desired communications 
environment can be established. In comparison with a format 
of the prior art wherein communications connection control 
icons endowed with fixed functions which have been configured 
in correspondence to communications media are operated and 
wherein multiple types of communications media which are 
necessary for the objective conferences or joint operations 
are individually connected in a controlled state, therefore, 
cumbersome repetitious routines can be eradicated, and the 
operations can be simplified, based on which flexible 
responses can be rendered in response to changes in the number 
of participants (number of terminals) of conferences or joint 
operations or to other environmental changes. 
(Application examples) 
[0012] 
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Figure 1 shows an electronic conversation system which 
consists of three work stations (40) (40-A, 40-B, and 40-C) , 
which are mutually coupled via the communications network 
(30), which may, for example, be instantiated by a LAN (local 
area network) or ISDN (integrated services digital network), 
etc. Each work station (40) is endowed with a multi-window 
function, and a window for displaying conference data to be 
used for the joint operation or electronic conversation is 
generated on each display screen J20) (20-A, 20-B, or 20-C)". 
Each work station (40), furthermore, is endowed with the 
telephone machine (10) (10-A, 10-B, or 10-C) , which 
complements the progress of a conference which is being 
monitored with reference to the display screen with audio 
means . 
[0013] 

Figure 2 is a block diagram which shows the fundamental 
constitution of the work station (40). In said figure, (140) 
is a memory deyice_ in which various programs and data are 
stored, whereas (90) is a central processing unit (CPU) 
which executes data processing routines in compliance with 
program commands decoded from the aforementioned memory device 
(140), whereas (100) is a bit map processor (BMP) which 
encodes contents to be outputted to the display screen (130) 
as image data in compliance with the command of the 
aforementioned CPU (90). The contents of the frame memory 
(110) are decoded by the display controller (120) and then 



displayed on the display screen (output device) (130) . 
[0014] 

(180-A) is a keyboard, which represents one of the input 
devices (180), whereas (160) is a keyboard driver (160) [sic: 
Duplication] which stores the code data inputted via said 
keyboard into the register (150-A), whereas (180-B) is a 
mouse, which represents another of the input devices (180), 
whereas (170) is a mouse driver (170) [sic: Duplication] which 
stores the code data inputted via said mouse into the register 
(150-B) . 
[0015] 

Each work station (40) is connected to the communications 
network (30), which is constituted by a LAN , ISDN, etc. via 
the communications interface device (80) . Incidentally, in a 
case where the ISDN is used as the communications network 
(30), the output of the telephone machine (10) is outputted to 
the communications network (30) via the communications 
interface device (80). In this application example, the 
telephone machine (10) can be initialized by the CPU (90). 
[0016] 

Figure 3 shows one example of the operation panel window 
(300), which is displayed on the display screen (20) of the 
display (130) of each work station for controlling the multi- 
media communications connection. Three types of icons, namely 
the telephone icon (310), TV telephone icon (320), and the 
conference icon (330), are prepared for the aforementioned 
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operation panel window (300) as communications control icons, 
and the. address book icon (340) is additionally prepared for 
it for designating connection nodes. In this example, a state 
where the address book window (350) has been opened on the 
display screen by the user's selection of the address book 
icon (340) is shown. 
[0017] 

In a case where a joint operation is initialized anew, 
confirmation telephone calls are first placed to other 
participants. In this case, the mouse (180-B) is operated for 
selecting the telephone icon_ (310) on the operation panel 
window (300) first and for subsequently selecting the address 
book icon J340), followed by the display of the address book 
window (350) . This selection order, however, may also be 
reversed. 
[0018] 

The address book window (350) includes a comprehensive 
list of members which can be called upon to participate in the 
joint operation, and in a case where a user selects the names 
of prospective participant members from the aforementioned 
comprehensive list via the mouse (180-B), preliminarily 
memorized telephone numbers are first dialed automatically, as 
a result of which calls arrive at the members. The connection 
of the voice line is completed upon the pickup of the call 
recipient's receiver. In this case, the terminal addresses of 
the members who have been selected on the address book window 



(350) are automatically memorized as joint operation 
participants, and in a case where connection control routines 
for other media (data and video images) are subsequently 
executed in relation to the aforementioned participant 
members, there is no need to redesignates the same members 
once again on the address book window (350) . The 
aforementioned address book icon (340) is also utilized 
according to procedures identical to the aforementioned one in 
a case where new members are added to the environment of a 
joint operation which has already begun. 
[0019] 

Instead of being placed within the special operation 
window (300), the telephone icon (310), TV telephone icon 
(320) , and conference icon (330) may also be configured within 
the window (420-A), which is used for individual operations 
(local operations), which will be discussed later with 
reference to Figure 6. The attributes of these icons are 
managed by the icon status management table (500), as will be 
discussed later with reference to Figure 7. The functions of 
these icons vary in accordance with adventitiously achieved 
statuses . 
[0020] 

Figure 4 shows one example of the display screen of the 
display screen (20) of a case where three members participate 
in a joint operation. In addition to the telephone icon 
(310), TV telephone icon (320), conference icon (330), and 



address book icon (340) , which have been shown with reference 
to Figure 3, as well as the participant buttons (360) (360-A 
and (360-B) are hereby displayed. 
[0021] 

In a case where the joint operation is performed by two 
participants (i.e., pervasion of only one other node), only 
one address needs to be memorized, whereas in a case where two 
or more other nodes exist, two or more addresses must be 
correspondingly memorized. In a case where a command is 
issued in relation specifically to one of multiple other 
participants, the node in question is specified by using said 
participant buttons (360) . 
[0022] 

The display areas (370) (370-A and 370-B) , which show the 
types of communications media through which the stations of 
the participants indicated by the respective buttons are 
connected to ones' own selection, are configured in the 
vicinity of the aforementioned participant buttons (360) . In 
the case of the example shown in the figure, three types of 
communications media, namely the data system ("D") , sound 
system ("V"), and video system ("S") [sic: "V" and "S" should 
presumably be swapped] , are connected to the station of the 
participant "Shimazaki" (360-A), whereas two types of 
communications media, namely the data system and sound system, 
are connected to the station of the participant "Kobayashi." 
[0023] 
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In the context of coordinating or communally sharing the 
application programs (APs) of the "datum 1," which is 
indicated by the icon (400-A) , and the "datum 2," which is 
indicated by the icon (400-B), on the conference room window 
(390) , the data communications media enable the communications 
of input commands or data related to these APs among the 
respective stations. The sound communications medium promotes 
an unhindered joint operation by communicating the sound 
signals to the respective stations via telephone machines. 

The video image communications medium transmits the images of 

j re* /-- f ' r> 

users' faces which have been picked up by TV cameras ! * 

auxiliarily attached to the respective stations to other 

stations in such a way that the facial expressions of other 

participants can be mutually observed while being displayed, 

for example, on the TV telephone window (380). In the 

aforementioned example, the conference room window (390) is 

shared by three members, namely "Shimazaki," "Kobayashi," and 

oneself. No video image medium, however, is connected with 

the participant "Kobayashi," and therefore, no TV telephone /5 

window for the participant "Kobayashi" is opened on the 

screen. In a case where one wishes to display the facial 

image of the participant "Kobayashi" in this state, namely in 

a case where one wishes to establish a communications line of 

the video image medium in relation to the "Kobayashi" station, 

the TV telephone icon (320) is selected on the operation panel 

window (300), and the participant button (360-B) , which 



corresponds to the participant "Kobayashi," is subsequently 
selected. As an alternative mode of operation, furthermore, 
the communications network (30) may be picked and then moved 
onto the participant button (360-B) for "Kobayashi." A line 
connection control routine indicated by the TV telephone icon 
(320) automatically establishes a video image communications 
line vis-a-vis the station of the participant " Kobayashi in 
response to these operations. 
[0024] 

Figure 5 shows one example of the display screen of the. 
display screen (20) in a case where one engages in a joint 
operation with two other participants, as Figure 4 indicates. 
The participant statuses and participant-specific 
communications media statuses are identical to their 
counterparts in Figure 4. 
[0025] 

A case where the operation panel windows (410) (410-B and 
410-C) are configured for the respective participants is 
hereby shown as a modified example of the operation panel 
window (300) pertaining to the case of Figure 4. The 
participant buttons (360) can be dispensed with by thus 
configuring participant-specific operation panel windows (410) 
and by memorizing the device addresses of the corresponding 
participants into the respective panel windows. In other 
words, in a case where the telephone icon (310), TV telephone 
icon (320), and the conference icon (330) are selected on the 



operation panel windows (410) {410B or 410C), the node device 
addresses corresponding to the respective panel windows can be 
procured. 
[0026] 

As far as the aforementioned format wherein participant- 
specific operation panel windows are configured is concerned, 
the operation panel window (410-A) , which serves as a master, 
is preliminarily prepared, and a new participant is designated 
by operating this master panel window, and the participant- 
specific operation panel window (410), which corresponds to 
said participant, is thus generated. There is no need to 
configure the address book icon (340) on the participant- 
specific operation panel window (410), and it needs to be 
configured only on the master panel window (410-A) or 
individual operation window (420) . The designation of the new 
member is executed by generating the address book window (350) 
by operating the aforementioned address book icon (340) and by 
designating the objective participant on its comprehensive 
list. 
[0027] 

Only one operation panel window (300) suffices for the 
format shown in Figure 4 regardless of the number of 
participants, based on which the number of windows to be 
displayed on-screen for the joint operation can be minimized. 
In this respect, the format shown in Figure 5 may be said to 
be problematic in that the number of windows to be displayed 
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o^-screen increases as the number of participants increases, 
although it is conversely advantageous in that an operation 
for designating a prospective participant (s) via the 
participant button (360) upon each redesignation of the 
communications media modes can be dispensed with. As far as 
actual applications are concerned, it is also feasible for a 
user to select either of the aforementioned two formats. 
[0028] 

Figure 6 shows one example of the display screen (20) on 
which the connection request confirmation window ( 430) has 
been generated. The connection request confirmation window 
(430) may, for example, be generated for establishing a 
connection upon the approval of the user who has received the 
request in a case where a connection request for 
communications media other than the sound system has been 
received from a node station, as in the case of a conference 
room connection request, for example. In other words, an 
inconvenient uninvited arising of a joint operation 
environment (e.g., conference room window (390), etc.) on the 
screen (20) of the work station as a result of an operation of 
the other party while a user engages in a data processing 
routine via the individual operation windows (420-A) and (420- 
B) may be avoided by the preliminary output of the inquiry 
message (440) for the user into the connection request 
confirmation window (430) and by selecting the "YES" button ' 
(450) in a case where the user can accept the connection of 
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the conference room [window] (390) or by selecting the "NO" ■ 
button (460) in a case where the user wishes to decline the 1 
connection request. In a case where preliminary agreements 
have been obtained via telephone connections, etc., 
furthermore, the display of the connection request 
conformation window (430) may also be inhibited [sic: 
Presumably "disabled"] for further simplifying the operation. 
[0029] 

Figure 7 shows the icon functional status table (500), 
which serves a function of managing the statuses of the media 
icons (i.e., telephone icon (310), TV telephone icon (320), 
and conference icon (330) ) on the operation panel window 
(300) . One example of set of standard setting designations at 
the initial stage is indicated by the table shown in the 
figure . 
[0030] 

The communications media (502) which must be connected or 
established upon the selections of their respective icons are 
defined in correspondence to the media icon names (504). In a 
case where the telephone icon (310) has been selected at the 
initial stage, for example, only the sound system is 
characterized by the "ON" status, and accordingly, the sound 
system alone becomes connected. In a case where the TV 
telephone icon (320) has been selected, both the sound and 
video image systems, which are characterized by their 
respective "ON" statuses, become connected, whereas in a case 
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where the conference icon (330) has been selected, all the 
sound, video image, and data systems, which are characterized 
by their respective n ON" statuses, become connected. 
[0031] 

The aforementioned icon functional status table (500) is 
configured for each participant, namely for the connection 
node address. In the case of Figure 4, the table (500) is 
configured for each participant button (360) on the operation 
panel window (300), whereas in the case of Figure 5, the table 
(500) is configured for each- participant-specific operation 
panel window. 
[0032] 

The designated functions of the aforementioned icon 
functional status table (500) become updated from its initial 
settings on every media icon selection occasion. After the 
telephone icon (310) has already been selected, for example, 
the sound system is already connected, and therefore, the 
status of the icon functional status table (500) varies 
according to Figure 8. Incidentally, the designation contents 
of the table initial settings can be altered by users' 
operations . 
[0033] 

Figure 8 shows the icon functional status table (500'), 
which has come to prevail as a result of the status change of 
the icon functional status table (500) as a result of the 
selection of the telephone icon (310) . Sound systems are 



already connected to the respective participants corresponding 
to the icon functional status table via telephone icons, and 
therefore, the sound system of the communications media 
becomes characterized by an "OFF" status in relation to each 
of the telephone icon (310), TV telephone icon (320), and 
conference icon (330) on the updated icon functional status 
table (500'). In a case where an operation for connecting the 
communications path with the participants corresponding to the 
table (500') is carried out anew by using these icons, 
therefore, [sic: No predicate] ... in a case where the TV 
telephone icon (320) is used, for example, the video image 
system alone becomes connected while the sound system is being 
excluded. 
[0034] 

Figures 9 through 13 show medium-specific logic 
communications path connection modalities among participants. 
The logic communications path connection modality of the sound 
system (540) upon the selection of the telephone icon (310) on 
the operation panel window (300) with regard to two members, 
namely the participants (A510) and (B520) , is shown in Figure 
9. In a case where the logic communications path for the 
sound system (540) becomes connected, connection standby 
statuses vis-a-vis the data system (550) and the video image 
system (560) are assumed by the respective sites in 
preparation for the next connection command. 
[0035] 



Figure 10 shows a state where the data system (550) and 
video image system (560) have been further connected based on 
the selection of the conference icon (330) on the operation 
panel window (300) in the sound system connection state shown 
in Figure 9. 
[0036] 

Figure 11 shows a state where the state where the two 
participants, namely (A510) and (B520), are connected via the 
sound, data, and video image systems is expanded into a three 
member connection state where the participant (C530) has been 
ackled anew. An operation for the participant (A510) to select 
the telephone icon (310) on the operation panel window (300) 
and a node selection operation on the address book window 
(350) are hereby carried out in the context of calling the new 
participant (C530) , and a case where the cyclic sound system 
(540) connection configuration has been achieved among the 
three participants as a result of the reception of the 
telephone call by the participant (C530) is hereby shown. In 
a case where the sound system connection is thus established, 
the participant (C530) assumes a communications path standby 
status. The data system (550) and video image system (560) 
still remain connected with only two participants, namely 
(A510) and (B520), at this stage. 
[0037] 

Figure 12 shows a state where the data system (550) and 
video image system (560) have been connected to the 
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participant (C530) as a result of the operation of the 
conference icon (330) by the participant (A510) shown in 
Figure 11. The participant (A510) engages in an operation for 
connecting the data system and video image system anew to the 
participant (C530) , who is standing by for connection paths, 
and subsequently, the communications paths of the data system 
(550) and video image system (560) to the participant (B520) 
are severed. The participant (B520) assumes a standby status 
awaiting the next connection as soon as these communications 
paths become severed. The participant (C530) is notified of 
the address of the participant (B520) , and it is commanded to 
connect the communications path to the participant (B520) . 
[0038] 

Figure 13 shows a state where the participant (C530), who 
has received the command from the participant (A510) , has 
connected the data system (550) and video image system (560) 
to the participant (B520) , who has been standing by for new 
communications paths shown in Figure 12. As the figure 
indicates, a logic communications path of the sound system 
(540), data system (550), and the video image system (560) 
becomes formed in a cyclic fashion among the three 
participants (A510) , (B520), and (C530) . 
[0039] 

Figure 14 shows the processing flow of a case where the 
participant (A510) has operated the TV telephone icon (320) on 
the operation panel window (300) in a state where no 



connection whatsoever has been established between two 

participants (A510) and (B520) . 

[0040] 

^Step 1 (606): The operation 1 by the participant (A510) 
(602) and the operation 2 by the participant (B520) (604) are 
executed. The participant (A510) executes the operation 1 
(602) by selecting the TV telephone icon (320) on the 
operation panel window (300) while the address book icon (340) 
is being concomitantly selected, by displaying the address 
book window (350), and by designating the participant (B520) 
from the address list. As a result, a telephone call is 
transmitted to the participant (B520) . The participant (B520) 
executes the operation 2 (604) by picking up the receiver and 
by accepting the sound connection request from the participant 
(A[510]). 
[0041] 

..^ Step 2 (608): A state where the two participants are 
mutually connected via the sound system as a result of the 
aforementioned operation is shown. 
[0042] 

-'Step 3 (610): A state where both the participants (A510) 
and (B520) have come to assume communications path standby 
statuses is shown. The participant (A510) retains the address 
for the participant (B520) upon the designation of the 
participant (B520) in correspondence to the operation panel 
window (300) . 
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[0043] 

/ Step 4 (612) : A state where the video image system has 
been connected from the participant (A510) to the participant 
(B520) based on the address retained above is shown. The 
connection action based on the function of the TV telephone 
icon (320) is completed upon the connections of the sound 
system and video image system. 
[0044] 

Figure 15 shows a processing flow during a transition to 
a conference upon the connection of the sound system from a 
state where the two participants (A510) and (B[520]) are 
utterly unconnected. 
[0045] 

'Step 1 (626): The participant (A510) executes the 
operation 1 (620) by selecting the telephone icon (310) on the 
operation panel window (300) while the address book icon (340) 
is being concomitantly selected and displayed on the address 
book window (350) and by designating the participant (B[520]) 
from the address list. As a result, a telephone call to the 
participant (B520) becomes transmitted. The participant 
(B520) executes the operation 2 (604) by picking up the 
receiver and by accepting the sound connection request from 
the participant (A[510] ) . 

[0046] 

i 

Step 2 (628) : A state where the sound [system] has been 
connected between the two participants as a result of the 
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aforementioned operation is shown. 
[0047] 

■ Step 3 (630) : Both the participants (A510) and (B520) 
come to assume communications path standby statuses. The 
participant (A510) retains the address of the participant 

(B520) based on the designation of said participant (B520) . 

[0048] 

. Step 4 (632) : The participant (A510) executes the 
operation 3 (624) by selecting the conference icon (330) 
on the operation panel window (300) . The function of the 
conference icon (330) at this stage is determined with 
reference to the icon functional status table (500) . Since 
the sound system has already been connected at step 1 (626), 
the conference icon (330) can be functionally characterized as 
follows: Sound: "OFF" ; video image: "ON"; data: "ON." 
[0049] 

-Step 5 (634): The participant (A510) transmits a video 
image connection request and a data connection request 
addressed to the participant (B520) based on the address of 
the participant (B520) retained at step 3. The connection 
request confirmation window (430) shown in Figure 6 is 
concomitantly displayed on the work station screen (20) of the 
participant (B520) . 
[0050] 

Step 6 (636): In a case where the participant (B520) 
inputs "YES" into the window (430), the video image connection 



./request and data connection request from the aforementioned 
participant (A510) are accepted. As a result, communications 
paths of the data and video image system are established 
between the two participants in addition to the previously 
established sound system. 
[0051] 

In a case where the video image system (560) becomes 
connected, the TV telephone window (380) comes to be displayed 
on the work station screen (20) by initializing a program 
which controls the TV telephone window (380) or by executing a 
new routine, whereas in a case where the data system becomes 
connected, the conference room window (390) comes to be 
displayed on the work station screen (20) by initializing a 
program which controls the conference room window (390) or by 
executing a new routine. As a result, a joint operation which 
utilizes sounds exchanged through the telephone line, video 
images exchanged through TV telephone systems, and AP output 
data displayed on the conference room window (390) becomes 
possible for the two participants. 
[0052] 

Figure 16 shows a processing flow during a transition 
from a state where a conference is being convened between the 
two participants (A) and (B) (while they are being connected 
via the sound system, data system, and the video image system) 
to a state where a three-member conference is convened by 
adding the participant (C) . 



[0053] 

Step 1 (646): The participant (A510) and participant 
(B520) are already connected via the sound system, data 
system, and the video image system. 
[0054] 

Step 2 (648): The participant (A510) executes the 
operation 1 (640) by selecting the TV telephone icon (320) on 
the operation panel window (300), and after the address book 
window (350) has been displayed by selecting the address book 
icon (340), the participant (C[530]) is designated from the 
address list. As a result, a telephone call is transmitted to 
the participant (C530) . 
[0055] 

Step 3 (650): The participant (C530) executes the 
operation 2 (642) by picking up the receiver and by accepting 
the sound connection request from the member (A[510]). The 
participant (A510) comes to retain the address of the 
participant (C530) at the aforementioned step where the 
participant (C530) has been designated. 
[0056] 

Step 4 (652): The three participants (A), (B) , and (C) 
become mutually connected via the sound system as a result of 
the aforementioned operations of 1 and 2. 
[0057] 

Step 5 (654): As soon as the aforementioned sound system 
connection has been established, the participant (C530) comes 



to assume a communications path standby status in preparation 

for connections via other communications media. 

[0058] 

Step 6 (656) : The participant (A510) executes the 
operation 3 (644) by selecting the conference icon (330) on 
the operation panel window (300) and by selecting the 
participant (C[530]) via the participant button (360). The 
icon functional status table (500-C) , which corresponds to the 
participant (C530) , is already connected to the sound system 
at this stage, and therefore, the conference icon (330) is 
functionally characterized by "ON" statuses with regard to the 
video image and data among the communications media. That the 
conference icon (330) is executed in relation to the 
participant (C530) at this stage therefore signifies the 
connections of the video image system and data system. 
[0059] 

[S]tep 7 (658): The participant (A510) severs the 
communications paths of the video image system and data system 
toward the participant (B520) . The participant (B520) , the 
communications paths of the video image system and data system 
to which have been thus severed, assumes a communications path 
standby status in preparation for re-connections of these 
communications paths. 
[0060] 

Step 8 (660): The participant (A510) submits a video 
image system and data system communications path connection 
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request to the participant (C530) . 
[0061] 

Step 9 (662): The participant (C530) inputs "YES" on the 
connection request confirmation window (430) . As a result, 
the data system and video image system communications paths 
from the participant (510A) to the participant (C530) become 
connected. 
[0062] 

Step 10 (664) : The participant (A510) issues a command 
for connecting the video image system and data system 
communications paths vis-a-vis the participant (B520) to the 
participant (C530) together with the address of the 
participant (B520) . 
[0063] 

Step 11 (668): The participant (C530) retains the 
aforementioned address of the participant (B520) . 
[0064] 

Step 12 (670) : The participant (C530) establishes video 
image system and data system communications paths with the 
participant (B520) based on the address of the participant 
(B520) , which has been notified by the participant (A510) . 
[0065] 

It is thus that a routine for connecting the 
communications paths of the sound system, data system, and the 
video image system in a cyclic fashion among the three 
conference participants becomes completed. 



[0066] 

Figure 17 is a flow chart pertaining to the routine of 
the program (680), which executes the multi-media 
communications connection of the present invention. 
[0067] 

The communications system which is initially connected by 
the present program (680) is a telephone-based sound 
communications path (step 682). Upon the connection of the 
sound communications path, it is judged whether or not the 
respective communications paths of the other communications 
media are characterized by standard statuses (step 684). In a 
case where [any of] said communications paths fail[s] to be 
characterized by a standby state, the prevailing status is 
permutated into a communications path standby status (step 
686), whereas in a case where said communications path is 
characterized by a standby status, an advancement is made 
directly to the next step 688 directly, where an input standby 
status is assumed. 
[0068] 

Concrete examples of inputs awaited at step 688 include a 
user's input into the operation panel window (300) and a 
connection request received from another site. Upon the 
detection of any input, it is judged whether or not said input 
represents the severance of the sound (step 690) . The sound 
severance occurs as a result of the replacement of the 
telephone receiver, and in a case where a user replaces the 
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receiver, the communications path for the sound system becomes 
instantly severed. Upon the severance of the sound, it is 
first judged whether or not connections are being perpetuated 
with the other communications media (data system and video 
image system) (step 692), and in a case where the connection 
with any of these communications media is acknowledged, 
requests for severing its communications path are transmitted 
to other participants (step 694) while the communications 
path(s) which is still being connected is being severed (step 
696) , as a result of which the routine is completed (step 
698) . In a case where none of the communications paths other 
than that for the sound system have been judged to be 
connected at step 692, the routine is completed at this stage 
(step 698). In a case where the input is judged to be other 
than the severance of the sound at step 690, furthermore, the 
input routine explained below (step 700) is executed. 
[0069] 

Figure 18 is a flow chart which shows the input routine 
(700) in detail. In the following, the contents of the 
aforementioned input routine will be explained with reference 
to an example of operation panel window (300) shown in Figure 
4, on which multiple participants are simultaneously 
displayed. 
[0070] 

First, it is judged whether or not the prevailing input 
concerns an icon selection on the operation panel window 
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(300) (step 702). In a case where it is judged to be other 
than the input into the operation panel window (300), it is 
judged whether or not it is a connection request from another 
site (step 734). In a case where it has been judged to be a 
connection request from another site, the connection request 
confirmation window (430) is displayed on the work station 
screen (20) (step 736), and it is judged whether or not "YES" 
has been inputted into this window by the user (step 738) . In 
a case where the input of "YES" has been acknowledged, the 
communications path which is the object of the connection 
request is connected (step 740), and subsequently, the 
contents of the icon functional status table (500) are renewed 
(step 744). In a case where the input of "NO" has been 
acknowledged at the judgment step 7 38, the routine is 
completed at this stage. In a case where the prevailing input 
is judged to be other than a connection request from another 
site at the judgment step 734, furthermore, a processing 
routine corresponding to said input is executed (step 742). 
[0071] 

In a case where the input has been judged to be destined 
to the operation panel window (300) at the judgment step 702, 
it is checked whether or not the number of participants who 
have been previously connected via communications paths is 3 
or larger. The number of participants is memorized into this 
program as a variable (e.g., N), and whenever a new 
participant joins the joint operation, 1 each is added to the 



prevailing variable N. A case where the number of 
participants is assumed to be 2 will be explained below. 
[0072] 

That the number of participants is 2 means that there is 
only one other conference participant, and therefore, there is 
no need to designate the participant button (360) on the 
operation panel window (300) . In other words, the address 
which is being retained in correspondence to the operation 
panel window (300) inevitably belongs to the only other 
conference participant, and in a case where the telephone icon 
(310), TV telephone icon (320), or conference icon (330) is 
selected on this operation panel window (300), the address of 
the other participant is monolithically determined. In this 
example, therefore, the address of the other participant is 
obtained after an advancement has been made to step 720 from 
the judgment step 704. 
[0073] 

Next, the icon functional status table (500) 
corresponding to the aforementioned participant is referred to 
(step 722), followed by the acquisition of a function 
corresponding to the selection media icons defined in said 
table (step 724) and then by the execution of said function 
(step 726) . Next, a connection request is transmitted to a 
site which serves as a connection node (step 728), and in a 
case where "YES" has been obtained from the other party in 
response to the aforementioned connection request, the 
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connection routine is completed (step 732), and after the 
function designation status of the icon functional status 
table (500) corresponding to the connection node has been 
updated (step 744), this routine is power-corrected [sic: 
Presumably "completed?"]. In a case where "NO" is obtained as 
a response at step 730, this routine is completed without 
establishing a connection. 
[0074] 

In a case where the number of participants has been 
judged to be 3 or larger at the judgment step 704, it is 
judged whether or not the participant button has been pressed 
on the operation panel window (300) (step 706) . In a case 
where said participant button has been pressed, a participant 
address corresponding to the aforementioned participant button 
is obtained (step 718), and a connection with the other party 
is attempted through the aforementioned routine from step 722 
to step 744. 
[0075] 

In a case where the participant button (360) has been 
judged to have not been pressed at the judgment step 706, the 
following routine is executed. That the participant button 
(360) has not been pressed after the selection of a media icon 
(telephone icon (310), TV telephone icon (320), or conference 
icon (330)) means that no call has been placed to the new 
participant. In this case, therefore, the selection of the 
address book icon (340) is awaited (step 708), and in a case 
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where the address book icon (340) has been selected, the 
address book window (350) is displayed on the work station 
screen (20) (step 710) . Next, the input of the address of the 
other party is awaited (step 712), and in a case where the 
input of the selection of the address of the other party has 
been acknowledged, the address of the other party is obtained 
and retained (step 714). Upon the. completion of this stage, 
the routine corresponding to steps 722 and beyond is executed, 
and a communications path connection with the other site is 
attempted. 
[0076] 

(Effects of the invention) 

As the foregoing explanations have clearly demonstrated, 
as far as the multi-media terminal and multi-media 
communications connection control format of the present 
invention are concerned, communications connection control 
icons are endowed with dynamically variable functions, and 
therefore, operations for redesignating communications media 
connection statuses can be simplified by invoking different 
functions from an identical icon. After the reception side 
has picked up the telephone receiver, furthermore, the 
objective communications environment can be established easily 
based on operations on the transmission side alone. 
Brief explanation of the figures 

Figure 1: A diagram which shows the overall constitution 
of a conference system to which the present invention can be 



applied. 

Figure 2: A diagram which shows the hardware constitution 
of a terminal device (work station) which constitutes [said] 
conference system. 

Figure 3: A diagram which shows one example of the 
operation panel window (300) of the present invention to be 
displayed on a work station screen. 

Figure 4: A diagram which shows one example of the 
operation panel window (300) in a case where multiple 
participants are involved. 

Figure 5: A diagram which shows one example of the 
participant-specific operation panel windows (410) which 
represents a variation of the aforementioned operation panel 
window (300) . 

Figure 6: A diagram which shows one example of the 
connection request confirmation window (430) which is 
displayed on the work station screen. 

Figure 7: A diagram which shows one example of the icon 
functional status table (500) which the multi-media 
communications connection program (680) possesses. 

Figure 8: A diagram which shows the variation of the 
functional status of the icon functional status table as a 
result of an icon operation. 

Figure 9: A diagram which shows a state where two 
conference participants have become mutually connected via a 
sound system logic communications path. 
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Figure 10: A diagram which shows a state where the logic 
communications paths of data and video image systems are 
additionally connected in the aforementioned state of Figure 
9. 

Figure 11: A diagram which shows a state where a sound 
system logic communications path has become established in a 
case where a third participant has joined a conference between 
two participants. 

Figure 12: A diagram which shows a process for 
establishing the communications paths for the data and video 
image systems in the state of Figure 11. 

Figure 13: A diagram which shows a state where sound 
system, data system, and video image system logic 
communications paths have become connected among three 
participants in a cyclic fashion. 

Figure 14: A demonstrational diagram pertaining to a case 
where the participant (A) has operated the TV telephone icon 
(320) in a state where no connection is being established 
between two participants. 

Figure 15: A demonstrational diagram pertaining to a 
process for connecting the video image system and data system 
by operating the conference icon (330) in a state where the 
two participants are mutually connected via a sound circuit. 

Figure 16: A demonstrational diagram pertaining to the 
flow of a process whereby a transition is made from a two- 
member conference to a three-member conference. 
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Figure 17: A flow chart which shows the program (680) for 
the mult i -media communications connection of the present 
invention. 

Figure 18: A flow chart which shows the input routine 
(700) of Figure 17 in detail. 
(Explanation of notations) 

(10): Telephone machine; (20): Work station; (30): 
Network; (50) : Telephone switchboard network; (60) : Individual 
utility window; (70) : Joint utility window; (130) : Output 
device;, (180) : Input device; (190) : Window system; (300) : 
Operation panel window; (310) : Telephone icon; (320) : TV 
telephone icon; (330) : Conference icon; (340) : Address book 
icon; (350) : Address book window; (360) : Participant button; 
(370): Display area; (380): TV telephone window; (390): 
Conference room window; (400): Data icon; (410): Participant- 
specific operation panel windows; (420) : Individual operation 
window; (430) : Connection request confirmation window; (440) : 
Connection confirmation message; (500) : Icon functional status 
table; (502) : Communications media; (504) : Media icon names; 
(540) : Sound system; (550) : Data system; (560) : Video image 
system; (680): Multi-media communications connection program; 
(700) : Input routine for the multi-media communications 
connection program. 
Keys to the Figures 
Figure 3 

[(A): Telephone; (B) : TV telephone; (C) : Conference; (D): 



Address book; (E) : Kobayashi; (F) : Shimazaki; (G) : Tamiya; 
(H) : Akutagawa] 
Fiaure 4 

[(A): Telephone; (B) : TV telephone; (C) : Conference; (D) : 
Address book; (E) : Shimazaki; <F) : Kobayashi; (G) : Data/video 
image/sound; (H) : Data/sound; (I): Conference room; (J): Datum 
1; (K) : Datum 2] 
Fiaure 1 

[(10-A), (10-B), and (10-C) : Telephone machines; (20-A) , (20- 
C), and (40-B): Work station screens; (30): LAN or ISDN] 
Fiaure 5 

[(A): Shimazaki; (B) : Telephone; (C) : TV; (D) : Conference; 

(E): Kobayashi; (F) : Telephone; (G) : TV telephone; (H) : 
Sound/data; (I): Conference; (J): Master; (K) : Conference 
room; (L) : Datum 1; (M) : Datum 2; (N) : Individual operation; 

(O): Shimazaki; (P) : Data; (Q) : Figure; (R) : Memo; (S) : 
Address book] 
Fiaure 6 

[(A): Individual operation; (B) : Data; (C) : Figure; (D) : Memo; 
(E): Statistical chart; (F) : Connect to conference room?] 
Fiaure 7 

[(A): Telephone; (B) : TV telephone; (C) : Conference; (D) : 
Sound; (E): Video image; (F) : Data; (502): Communications 
media; (504) : Icon names] 
Fiaure 10 

[(540): Sound system; (550): Data system; (560): Video image 



system] 
Figure 2 



[(10): Telephone machine; (30): To LAN or ISDN; (80): 
Communications interface; (110): Frame memory; (120): Display 
controller; (130): Display; (140): Memory device; (150-A) and 
(150-B): Registers; (160): Keyboard driver; (170): Mouse 
driver; (180-A) : Keyboard; (180-B): Mouse] 
Figure 13 

[(540): Sound system; (550): Data system; (560): Video image 

system] 

Figure 8 

[(A): Telephone; (B) : TV telephone; (C) : Conference; (D) : 
Sound; (E) : Video image; (F) : Data; (502'): Communications 
media; (504'): Icon names] 
Figure 9 

[(540): Sound system; (X): Video image system and data system 
connection standby status] 
Figure 11 

[(540): Sound system; (550): Data system; (560): Video image 
system; (X) : Video image system and data system connection 
standby status] 
Figure 12 

[(540): Sound system; (550): Data system; (560): Video image 
system; (X) : Video image system and data system connection 
standby status] 



Figure 14 

[(X): Sound connection; (Y) : Address B; (Z) : Video image 
connection; (602): Operation 1; ( 604 ): Operation 2; (60b) : 
Step 1; (608): Step 2; (610): Step 3; (612): Step 4] 
Figure 15 

[(A): Sound connection; (B) : Address B; (C) : Video image 
connection request; (D) : Data connection request; (E) : Video 
image connection; (F) : Data connection; (620) : Operation 1; 
(622): Operation 2; (624): Operation 3; (626): Step 1; (628): 
Step 2; (630): Step 3; (632): Step 4; (634): Step 5; (636): 
Step 6] 
Figure 16 

[(1): Address B; (2): Address C; (3): Data connection; (4): 
Data connection request; (5) : Sound connection; (6) : Video 
image connection; (7) : Video image connection request; (640) : 
Operation 1; (642): Operation 2; (644): Operation 3; (646): 
Step 1; (648): Step 2; (650): Step 3; (652): Step 4; (654): 
Step 5; (656): Step 6; (658): Step 7; (660): Step 8; (662): 
Step 9; (664): Step 10; (668): Step 11; (670): Step 12] 
Figure 17 

[(680): Start; (682): Sound connection; (684): Communications 
path standby status?; (686) : Assumption of communications path 
standby status; (688): Input standby; (690): Sound severed?; 
(692): Communications path in connection?; (694): Telephone 
path severance request transmission; (696) : Communications 
path severance; (698): Complete; (700): Input routine; (E) : 
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End] 

Figure 18 

[(700): Input routine entry; (702): Icon input?; (704): Number 
of participants £ 3?; (706) : Participant button pressed?; 
(708) : Address book selected?; (710) : Address book display; 
(712) : Address book selected?; (714) : Acquisition of the 
address of the other party; (718): Acquisition .of . the address 
of the other party; (720) : Acquisition of the address of the 
other party; (722) : Reference to icon functional status table; 
(724) : Acquisition of icon function; (726) : Execution of the 
function; (728) : Connection request transmission; (730) : 
Reception of "YES"; (732) : Connection complete; (734) : 
Reception of connection request from the other site?; (736) : 
Display of connection request confirmation window; (738) : 
"YES" or otherwise?; (740): Connection complete; (742): Other; 
(744) : Renewal of the iron functional status table] 
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